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1 Introduction 

Tensor-based methods are receiving a growing interest in scientific computing for the 
numerical solution of problems defined in high dimensional tensor product spaces, such 
as partial differential equations arising from stochastic calculus (e.g. Fokker-Planck 
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equations) or quantum mechanics (e.g. Schrodinger equation), stochastic parametric 
partial differential equations in uncertainty quantification with functional approaches, 
and many mechanical or physical models involving extra parameters (for parametric 
analyses),. . . . For such problems, classical approximation methods based on the a pri- 
ori selection of approximation bases suffer from the so called "curse of dimensionality" 
associated with the exponential (or factorial) increase in the dimension of approxi- 
mation spaces. Tensor-based methods consist in approximating the solution u £ V of 
a problem, where V is a tensor space generated by d vector spaces Vj (assume e.g. 
Vj = M""^ ]j, using separated representations of the form 

m 

u « u,„ = J2 ® • • • ® '^'i'^^ ' ^F'' e (1) 
1=1 

where (g) represents the Kronecker product. The functions w- are not a priori selected 
but are chosen in an optimal way regarding some properties of u. 

A first family of numerical methods based on classical constructions of tensor 
approximations [21"T7, 33 have been recently investigated for the solution of high- 
dimensional partial differential equations |18lf51 [^i,26j . They are based on the system- 
atic use of tensor approximations inside classical iterative solvers. Another family of 
methods, called Proper Generalized Decomposition (PGD) methods |25ll9l l31ll32lll5| . 
have been introduced for the direct construction of representations of type ([T|. PGD 
methods introduce alternative definitions of tensor approximations, not based on nat- 
ural best approximation problems, for the approximation to be computable without a 
priori information on the solution u. The particular structure of approximation sets 
allows the interpretation of PGDs as generalizations of Proper Orthogonal Decompo- 
sition (or Singular Value Decomposition, or Karhunen-Loeve Decomposition) for the 
a priori construction of a separated representation Um of the solution. They can also 
be interpreted as a priori model reduction techniques in the sense that they provide 
a way for the a priori construction of optimal reduced bases for the representation of 
the solution. Several definitions of PGDs have been proposed. Basic PGDs are based 
on a progressive construction of the sequence Um, where at each step, an additional 
elementary tensor ®^_]^Wm'* is added to the previously computed decomposition Um-i 
[24ll2l[28] . Progressive definitions of PGDs can thus be considered as Greedy algorithms 
[35j for constructing separated representations [6l[T]. A possible improvement of these 
progressive decompositions consists in introducing some updating steps in order to 
capture an approximation of the optimal decomposition, which would be obtained by 
defining the whole set of functions simultaneously (and not progressively). For many 
applications, these updating strategies allow recovering good convergence properties of 
separated representations [29II32II3T] . 

In [6], convergence results are given for the progressive Proper Generalized Decom- 
position in the case of the high-dimensional Laplacian problem. In ;15,, convergence 
is proved in the more general setting of linear elliptic variational problems in tensor 
Hilbert spaces. The progressive PGD is interpreted as a generalized singular value 
decomposition with respect to the metric induced by the operator, which is not neces- 

^ More precisely, V is the closure with respect to a norm || ■ || of the algebraic tensor space 
V = a Vj = span |(8)j^i v^^'^ : G Vj and 1 < j < d } 
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sarily a crossnorm on the tensor product space. 

In this paper, we propose a theoretical analysis of progressive and updated Proper 
Generalized Decompositions for a class of problems associated with the minimization 
of an elliptic and differentiable functional J, 

J(u) — min J(v), 

v<£V 

where V is a reflexive tensor Banach space. In this context, progressive PGDs consist 
in defining a sequence of approximations Um G V defined by 

where Si is a tensor subset with suitable properties (e.g. rank-one tensors subset, 
Tucker tensors subset, ...), and where Zm is an optimal correction in Si of u^-i, 
defined by 

^(Um-l + Zm) = min J(um_i + z) 

Updated progressive PGDs consist in correcting successive approximations by using 
the information generated in the previous steps. At step m, after having computed an 
optimal correction Zm £ Si of u^-i, a linear (or affine) subspace Um C V such that 
Um_i + Zm G Um is generated from the previously computed information, and the 
next approximation Um is defined by 

J{um) = min J(v) < J(u„i-1 +Zm) 

The outline of the paper is as follows. In section [2l we briefly recall some classical 
properties of tensor Banach spaces. In particular, we introduce some assumptions on 
the weak topology of the tensor Banach space in order for the (updated) progressive 
PGDs to be well defined (properties of subsets Si). In section O we introduce a class of 
convex minimization problems on Banach spaces in an abstract setting. In section |4l we 
introduce and analyze the progressive PGD (with or without updates) and we provide 
some general convergence results. While working on this paper, the authors became 
aware of the work [7], which provides a convergence proof for the purely progressive 
PGD when working on tensor Hilbert spaces. The present paper can be seen as an 
extension of the results of [7] to the more general framework of tensor Banach spaces 
and to a larger family of PGDs, including updating strategies and a general selection 
of tensor subsets Si. In section [SI we present some classical examples of applications of 
the present results: best approximation in tensor spaces (generalizing the multidi- 
mensional singular value decomposition to spaces), solution of p-Laplacian problem, 
and solution of elliptic variational problems (involving inequalities or equalities) . 



2 Tensor Banach spaces 

We first consider the definition of the algebraic tensor Space a ^j^i Vj generated from 
Banach spaces Vj (1 < J < d) equipped with norms || ■ As underlying field we choose 
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R, but the results hold also for C. The suffix 'a' in a ^'j=i '^j refers to the 'algebraic' 
nature. By definition, all elements of 



V:= a^V, 

i=i 

are finite linear combinations of elementary tensors v = ^j—i 'v^'^^ ^u^-'-' G Vj 
A typical representation format is the Tucker or tensor subspace format 

iel 3 = 1 

where I = Ji x . . . x is a multi-index set with Ij = {l,...,rj}, rj < dim(Vj), 
i)f^ € Vj {ij £ /j) are linearly independent (usually orthonormal) vectors, and a; £ M. 
Here, ij are the components of i = (ii, . . . ,i^). The data size is determined by the 
numbers rj collected in the tuple r :— (ri, . . . , r^;). The set of all tensors representable 
by ((2|) with fixed r is 

r there are subspaces Uj C Vj such that 1 

rr(V) |v e V : ^.^^^^^^ ^ and V G U . (g)^t, U, . / 

To simplify the notations, the set of rank-one tensors (elementary tensors) will be 
denoted by 

7^l(V) := 7^i,...,i)(V) = {®Ll™^'^ : ^^^'^ GVk}. 

By definition, we then have V — span7^i(V). We also introduce the set of rank-m 
tensors defined by 



We say that V||.|| is a Banach tensor space if there exists an algebraic tensor space 
V and a norm ||-|| on V such that V||.|| is the completion of V with respect to the 
norm ||-||, i.e. 

d 2 11-11 

V||.||-||.||(g)VS=a0 . 

J = l 

If Vjj.|| is a Hilbert space, we say that Vj|.|| is a Hilbert tensor space. 



2.1 Topological properties of Tensor Banach spaces 



Observe that span7?,i (V) is dense in V||.|| . Since TZi (V) C 7r(V) for all r > (1, 1, . . . , 1), 
then span7r(V) is also dense in V||.||. 

Any norm ||-|| on a ®'j=i satisfying 

= Yl^ ^ Ik \\j for aU v^^^ G Vj {1 < j < d) (4) 
is called a crossnorm. 
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Remark 1 Eq. (|4]) implies the inequality || (^j— i v^''^ \\ < 11^=1 ll^'^"''' II j which is equiv- 
alent to the continuity of the tensor product mapping 



aQ^^^, ,IMI I , 



(5) 



given by ® (^v^^\ . . . ,v^'^^)^ — ®j— where {X, \\ ■ ||) denotes a vector space X 
equipped with norm || ■ ||. 

As usual, the dual norm to || || is denoted by [|-[|*. If ||-|| is a crossnorm and also 



is a crossnorm on a , i-e 



= i- -3 
d 



n 



ip^-\\j for all ip'^^^ G V* (1 < j < d) , (6) 

is called a reasonable crossnorm. Now, we introduce the following norm. 

j for 1 < j < d. Then for 



Definition 1 Let Vj be Banach spaces with norms 
V £ V — a ^j—i Vj , we define the norm || ||^ by 



l|v|lv := sup ■ 



(d) 



(7) 



The following proposition has been proved in 114]. 



Proposition 1 Let V||.|| be a Banach tensor space with a norm satisfying || • || \\'\\\/ 
on V. Then the set 7r(V) is weakly closed. 



2.2 Examples 

2.2.1 The Bochner spaces 

Our first example, the Bochner spaces, are a generalization of the concept of L^-spaces 
to functions whose values lie in a Banach space which is not necessarily the space K or 

c. 

Let X be a Banach space endowed with a norm || • Let / C and /x a finite 
measure on I (e.g. a probability measure). Let us consider the Bochner space L^{T, X), 
with 1 <p< oo, defined by 

^i^v.I^X-. j^ \\vix)\\P^dfi{x) < c»| , 

and endowed with the norm 




We now introduce the tensor product space V||.||^ — X®||.||^ ^m(^)- ^"-"^ 1 < P < co, 
the space can be identified with V||.||^ (see Section 7, Chapter 1 in [10\). 

Moreover, the following proposition can be proved (see Proposition 7.1 in |1U|): 
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Proposition 2 For 1 < p < oo, the norm \\-\\/^^ satisfies \\-\\/^^ > ||-||y on X(g)ai^(/). 
By Propositions [2] and [TJ we then conclude: 

Corollary 1 For 1 < p < oo, the set Tr (E)a L^{I)^ is weakly closed in Lfi{I; X). In 
particular, for K cR'', we have that Tr (if! (K) ®a Lf,{I)) and TZi [Ll (K) ®a ^J.{I)) 
are weakly closed sets in L^^^ {K x I) . 

2.2.2 The Sobolev spaces 

Let i? = i7i X . . . X C K'', with fl^ C M. Let a e N'' denote a multi- index and 
\a\ = X]fe=i ^fc- 0"(u) — dxi . . . 9"^(u) denotes a partial derivative of u(a;i, . . . , xj) 
of order |a|. For a fixed 1 < p < oo, we introduce the Sobolev space 

= {u e LP{n) : i3"(u) e LP(«),0 < |a| < m} 

equipped with the norm 

Mm,p = Yl \\D"M\\LPin) 

0<\a\<m 

We let Vk = H™''P[Oi^), endowed with norms || • ||m,p:fc defined by 
Then we have the following equality 

d 

H'^'PiO) = ||.||_^^ (g)H'"'P(J7j) . 

A first result is the following. 
Proposition 3 For 1 < p < oo, m > and O — S7i x . . . x fi^j, the set 



7^l 



is weakly closed in (_ff™'^(J7), || ■ ||m,p)- 

To prove the above proposition we need the following two lemmas. 

Lemma 1 Assume I < p < oo and Q = QiX. . .xf2^. Then the set TZi ^ a ^'j=i ^^(^j) ^ 
is weakly closed in LP(J?). 

Proof Let {vn}„gN, with Vn = ®^=iVn \ be a sequence in TZi ®j=i L^iOj)^ that 
weakly converges to an element v £ LP(Q). Then the sequence {vn}„£N is bounded in 
LP{n), and also the sequences {^1^ C L'P{Qj) for each j G {1, 2, . . . , d}. Then, for 
each j £ {1,2, . . . ,d}, we can extract a subsequence, namely {vn^}^^^^, that weakly 
converges to some £ L^^Qj). Since weak convergence in ^{Qj) implies the con- 
vergence in distributional sense, that is, the subsequence }feGN converges to v^^^ 
in T>'{fij). From Proposition 6.2.3 of [4], we have that {®'j—iVnl'\k&i converges to 
in T)' {n). By uniqueness of the limit, we obtain the desired result. □ 
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Lemma 2 Assume 1 < p < oo, m > 1 and H — ili x . . . x n^i. For any measurable 
functions Wk : Ok R such that ^f—iW^ 7^ 0, we have (S>f—iWk £ H"^'^{Q) if and 
ifwkG for all k£{l...d}. 



Proof Suppose that wi^ £ H"'--P{f2f.) for all fc e {1 . . . d}. Since 

d 

0<\a\<m k=l 

d 

d / rn \ d 

= n Y\\^Xki^k)\\LP{nk) = n \\'^k\\m,p;k, 
k=l \j=0 J fc=l 

we have ^ 

Conversely, if ^f^iW^ e H"^'P{n), then 

II ®fc=l Wfc||m,p = E ll-C^"(®fc=l^fe)llLP(«) < °° 
0< |a| <m 

which implies that ||-D"(®fc— iWfc)||LP(f2) < for all a such that < jal < m. Taking 
a = (0, . . . , 0), we obtain 



t=l Wk\\Lp{f2) = n \\^k\\LP{i2k) < °o 



fc=l 

and therefore |[wfe llLP(i'2fc) < °o ^o'' ^- Now, for A; £ {1 . . . d}, taking a — {. . . ,0, j,0, . . .) 
such that aj. = j, with 1 < j < m, and a; = for I ^ k, we obtain 

ll^"(®f=i™i)llLp(fi) = l!9sfc™fcllLp(r2fc) n 

and then ||9^fc'!i'fe llj,,^^. < 00 for all j G {1, . . . ,m}. Therefore w^. £ H™''^{nfS) for all 
ke{l...d}. □ 

Proof of Proposition\B\ For m = Q the proposition follows from Lemma[T] Now, assume 
m > 1, and let us consider a sequence 

that weakly converges to an element z G i/™'^(J7). Since 



^1 c7^l |^a0i^'(«,)j , 

we have z e 7li ( a (8)^=1 ) because, from Lemma [T] the latter set is weakly 

closed in L^{Q). Therefore, there exist W}. G LP{Of^) such that z — ®f.—iW^,. Since 
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z e from Lemma H Wk G H"^'P{nk) for 1 < fc < d, and therefore z = 

From [14| it follows the following statement. 
Proposition 4 The set Tv (a<S>j=i is weakly closed in H'^''^{f2). 



3 Optimization of functionals over Banach spaces 

Let be a reflexive Banach space, endowed with a norm || ■ ||. We denote by V* the 
dual space of V and we denote by {■, •) : x — 5- R the duality pairing. We consider 
the optimization problem 

J{u) = min J{v) (tt) 
where J : 1/ — s- M is a given functional. 



3.1 Some useful results on minimization of functionals over Banach spaces 

In the sequel, we will introduce approximations of by considering an optimization 
on subsets M C V, i.e. 

inf J{v) (8) 

vGM 

We here recall classical theorems for the existence of a minimizer (see e.g. |13)). 

We recall that a sequence Vm £ is weakly convergent if limm^oo (<y5, 'Vm) exists for 
all (f £ V* . We say that (wm)„gpj converges weakly to u G V if limm-s-oo {<^, Wm) = {f, v) 
for all LP G V* . In this case, we write Vm v- 

Definition 2 A subset M C 1^ is called weakly closed if Vm G M and Vm v implies 
V G M. 

Note that 'weakly closed' is stronger than 'closed', i.e., M weakly closed =J> M 
closed. 

Definition 3 We say that a map J : V — > R is weakly sequentially lower semicon- 
tinuous (respectively, weakly sequentially continuous) in AI C V if for all G M and 
for all Vm G M such that Vm v, it holds J{v) < liminfrn->oo J{vm) (respectively, 
J{v) = lim„i^oo J(Vm)). 

If j' : 1/ — > V* exists as Gateaux derivative, we say that j' is strongly continuous 
when for any sequence Vn ^ v in V it holds that J'{vn) — >■ J'{v) in V* . 

Recall that the convergence in norm implies the weak convergence. Thus, J weakly 
sequentially (lower semi)continuous in M J (lower semi)continuous in M. It can be 
shown (see Proposition 41.8 and Corollary 41.9 in [37]) the following result. 



Proposition 5 Let V be a reflexive Banach space and let J : V ^ M. be a functional, 
then the following statements hold. 
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(a) If J is a convex and lower semicontinuous functional, then J is weakly sequentially 
lower semicontinuous. 

(b) If J' : V — > V* exists on V as Gateaux derivative and is strongly continuous (or 
compact), then J is weakly sequentially continuous. 

Finally, we have the following two useful theorems. 

Theorem 1 Assume V is a reflexive Banach space, and assume AI C V is bounded 
and weakly closed. // J : Af — > R U {00} is weakly sequentially lower semicontinuous, 
then problem ((S]) has a solution. 

Proof Let a — inf^jg^ J(«) and {vn} C A he a. minimizing sequence. Since A is 
bounded, {wnjngN is a bounded sequence in a reflexive Banach space and therefore, 
there exists a subsequence {vn^. }ki£ts that converges weakly to an element u £ V. 
Since A is weakly closed, u £ A and since J is weakly sequentially lower semicontin- 
uous, J(u) < lim inf;;_j.(X) JijirLk) = ot. Therefore, J{u) — a and u is solution of the 
minimization problem. □ 

We now remove the assumption that M is bounded by adding a coercivity condition 
on J. 

Theorem 2 Assume V is a reflexive Banach space, and M <Z V is weakly closed. If 
J : M ^ Ru{cx)} is weakly sequentially lower semicontinuous and coercive on M , then 
problem ((8| has a solution. 

Proof Pick an element wq € M such that J{vq) 7^ 00 and define Mq = {v £ M : J{v) < 
J{vq)}. Since J is coercive, Mq is bounded. Since M is weakly closed and J is weakly 
sequentially lower semicontinuous, Afp is weakly closed. The initial problem is then 
equivalent to inf^gjv/o which admits a solution from Theorem[T] □ 

3.2 Convex optimization in Banach spaces 

From now one, we will assume that the functional J satisfies the following assumptions. 

(Al) J is Frechet differentiable, with Frechet differential j' : V ^ V* . 
(A2) J is elliptic, i.e. there exist a > and s > 1 such that for all v,w £V; 

{j'{v) — j'{w),v — w) > a\\v — wW'^ (9) 

In the following, s will be called the ellipticity exponent of J. 

Lemma 3 Under assumptions (A1)-(A2), we have 

(a) Forallv,w£V, 

J{v) - J{w) > (J'{w),v- w) + -lit; - wf. (10) 

s 

(b) J IS strictly convex. 

(c) J IS bounded from below and coercive, i.e. lim||,„[|^oo J'(t') = +00. 
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Proof (a) For all v,w £ V, 



' d 



J{v)-J{w)^ / —J{w + t{v~w))dt= / {j' {w + t{v - w)) , V — w)dt 



'^^ Jo 
1 



{j'(w),v — w}+ / {j' (w + t{v ~ w)) — j' {w),v ~ w)dt 
Jo 

f 

■JO 

{j' (w),v — lu) H ||u — lull' 



>{j'{w),v~w)+ I j\\t(v~w)fdt 



(b) From (a), we have for v ^ w, 

J{v) ~ J{w) > {j'{w),v — 'w) 

(c) Still from (a), we have for all v £ V , 

j{v) > j(o) + {j'{o),v) + -\\vr > m - \\j'm\\\v\\ + -\\vr 



which gives the coercivity and the fact that J is bounded from below. 
□ 

The above properties yield the following classical result. 

Theorem 3 Under assumptions (A1)-(A2), the problem admits a unique solution 
u aV which is equivalently characterized by 

{J'{u),v)=0 \fveV (11) 

Proof We here only give a sketch of proof of this very classical result. J is continuous 
and a fortiori lower semicontinuous. Since J is convex and lower semicontinuous, it is 
also weakly sequentially lower semicontinuous (Proposition a)). The existence of a 
solution then follows from Theorem [21 The uniqueness is given by the strict convexity of 
J, and the equivalence between and Hll|l classically follows from the differentiability 
of J. □ 

Lemma 4 Assume that J satisfies (A1)-(A2). If {vm} C V is a sequence such that 
J{vm) — > J{u), where u is the solution of (jrj), then Vm — ^ u, i.e. 



\\u-Vm\\ — > 

m— f oo 

Proof By the ellipticity property (|10|) of J, we have 

J{Vm) - J(m) > {J'{u), Vm - u) + —\\u - Vm\f = —\\u - Vm\\^ . (12) 

8 S 

Therefore, 

— \\u-Vm\\''<J{Vm)-Jiu) > 0, 

S m— >oo 

which ends the proof. □ 
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4 Progressive Proper Generalized Decompositions in Tensor Banach 
Spaces 

4.1 Definition of progressive Proper Generalized Decompositions 

We now consider the minimization problem (j^ of functional J on a reflexive tensor 
Banach space V — V||.||. Assume that we have a functional J : V||.|| — > R satisfying 
(A1)-(A2) and a weakly closed subset iSi in V||.|| such that 

(Bl) Si C V, with G <Si, 

(B2) for each v £ iSi we have Av £ Si for all A G R, and 
(B3) span<Si is dense in V||.||. 

By using the notation introduced in Section [2.21 we give the following examples. 
Example 1 Consider V||.|| 

Example 2 Consider V||.|| 

Example 3 Consider V||.|| 

The set iSi can be used to characterize the solution of problem (j^ as shown by the 
following result. 

Lemma 5 Assume that J satisfies (A1)-(A2) and let u* £ satisfying 

J(u*) = min J(u* + z). (13) 

Then u* solves 

Proof For all 7 G R+ and z G 5i , 

J(u* +7z) > ^(u*) 

and therefore 

(J'(u*), z) = lim i(J(u* + 7z) - J(u*)) > 

7\,0 7 

holds for all z G <Si. From (B2), we have 

(/(u*),z) = Vzg5i, 

From (B3), we then obtain 

(J'(u*),v)=0 VvGV||.||, 
and the lemma follows from Theorem [3] □ 
In the following, we denote by Sm the set 

f m 

Sm = < ^ Zi ■■ Zi £ Si 

[i=l 

The next two lemmas will be useful to define a progressive Proper Generalized 
Decomposition. 



= LP (/; X) and Si = % [X ®a (I)) . 

= H"'^^{n) and Si = Tr (a (8),ti H"'-^{nj)) . 

= H'^^PiO) and Si = TZi [a^j^i H"'-P{n^)'^ . 
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Lemma 6 For each v £ ^he set 

V + 5i = {v + w : w G 5i} 

is weakly closed in V||.||. 

Proof Assume that v + Wn ^ w for some {w„}„>i C Si, then Wn ^ w — v and since 
Si is weakly closed, w — v G 5i. In consequence w G v + 5i and the lemma follows. 
□ 

Lemma 7 (Existence of a <Si-minimizer) Assume that J : V||.|| — > K satisfies 
(A1)-(A2). Then for any v G V||.||, the following problem admits a solution: 

min J(v + z) = min J(w) 

Proof Frechet differentiability of J implies that J is continuous and since J is convex, 
we have that J is weakly sequentially lower semicontinuous by Proposition O Moreover, 
J is coercive on V||.|| by Lemma |3Jc). By Lemma [S] v + iSi is a weakly closed subset 
in V||.||. Then, the existence of a minimizer follows from Theorem [S] □ 

Definition 4 (Progressive PGDs) Assume that J : V||.|| — > R satisfies (Al)- 
(A2), we define a progressive Proper Generalized Decomposition {uTn}m>l: over Si, 
of u = argmiUvgVii y Jiy) a^s follows. We let uq = and for m > 1, we construct 
Um G V||.|| from u,„_i G V||.|| as we show below. We first find an element Zm G iSi C V 
such that 

J(u,„_i + Zm) = min J{\im-l + z) (*)■ 

Then at each step m and before to update m to m + 1, we can choose one of the 
following strategies denoted by c, I and r, respectively: 

(c) Let Zm = Zm. Define Um = Um_i + Zm, update m to m + 1 and goto (*). 
(0 Let Zm — Ztti. Construct a closed subspace XJ(um_i + Zm) in such that 

Um_i + Zm G U(um-i + Zm). Then, define 

Um = arg min J(v), 

vGU(u„_i+z„,) 

update m to m + 1 and go to (*). 
(r) Construct a closed subspace U(zm) in V||.|| such that Zm G U(zm), and define 

Zm = arg min J(um.-i + z). 
zeu(z„.) 

Then, define 

Um = Um-i + Zm = arg min J(v), 

veu„_i+u(z„) 

update m to m + 1 and go to (*). 

Strategies of type (Z) and (r) are called updates. Observe that to each progressive 
Proper Generalized Decomposition {um}m>l of u we can assign a sequence of symbols 
(perhaps finite), that we will denote by 

a(u) = <y.iOL2 • ■ ■ OLk ' ' ' 
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where ct^. G {c,l,r} for all k — 1,2,.... That means that uj, was obtained without 
update if a^, = c, or with an update strategy of type i or r if a/- = Z or = r 
respectively. In particular, the progressive PGD defined in [7] coincides with a PGD 
where ocj^ — c for all k > 1. Such a decomposition is called a purely progressive PGD, 
while a decomposition such that cti^ — I or a/; — r for some k is called an updated 
progressive PGD. 

Remark 2 The update ocm = I can be defined with several updates at each iteration. 
Letting xi^ = u^-i + Zm, we introduce a sequence {um C V||.|| defined by 



Um = arg mm J(vJ 
veu(u<j:') 



with U(u^^) being a closed linear subspace of V||.|| which contains We finally let 



In [14 it was introduced the following definition. For a given v in the algebraic 
tensor space V, the minimal subspaces (7j,min(v) C Vj are given by the intersec- 
tion of all subspaces Uj C Vj satisfying v £ a ®'j—i Uj . It can be shown |14| that 
a ®'j—i t'j,min(v) is a finite dimensional subspace of V. 

Example 4 (Illustrations of updates) For a given Vm G ^||-|| i^-S- = u^-i + Zm if 
ctm = I OT Vm — im if OLm = r) there are several possible choices for defining a linear 
subspace U(vm). Among others, we have 

- U(vm) = a ®'j=i f{j',min(vm) . In the case of OLm = I, all subspaces U(Um_i +Zm) 
are finite dimensional and we have that Um £ V for all m > 1. 

- Assume that Vm = X^I^Li "^i^i some {zi, . . . ,Zm} C G M, 1 < i < m. 
Then we can define 

U(vm) = span {zi, . . . ,Zm}. 

In the context of Greedy algorithms for computing best approximations, an up- 
date of type OLm = r by using an orthonormal basis of U(vm) corresponds to an 
orthogonal Greedy algorithm. 

- Assume v,„ G V. Fix A: G {1, 2, . . . , d}. By using a 'S>'j=i Vj = Vfc (^a <S>j^k '^3 
we can write Vm = Yl^lLi ^'i'^ ® {®j^k '"i"'^) ^'^'^ some elementary tensors w']^^ ® 
{^®j^k ^F^^ ^"-"^ i = 1, . . . , m. Then we can define the linear subspace 



U(vm) = I ^ ® I ^w\'^ I : i;f ^ G Vfc, 1 < i < m 




The minimization on U(vm) corresponds to an update of functions along dimension 
k (functions in the Banach space Vj.). Following the remark[21 several updates could 
be defined by choosing a sequence of updated dimensions. 
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4.2 On the convergence of the progressive PGDs 

Now, we study the convergence of progressive PGDs. Recall that Zm £ 5i is a solution 
of 

J{Um-l + Zm) = niin J(Um_i + z), 
For OLm = c, we have Zm ~ Zm and u,n = u^-i + Zm, so that 

J{Um) = J(Um_i + Zm) = J{Ujn-l + Zm) 

For and Um is obtained by an update (or several updates) 

of u„j_i + Zm, so that 

J(u„i) < J(u,„_i + Zm) = J(u„i_i + Zm) 

Otherwise, for cxm = r, we have Um = Um-i + Zm with Zm obtained by an update of 
Zm , such that 

J{Um) = J(Um_i + Zm) < J{Um-l + Zm) 

We begin with the following Lemma. 

Lemma 8 Assume that J satisfies (A1)-(A2). Then {J{vim)}m>l, where {'Vim}m>l 
is a progressive Proper Generalized Decomposition, over Si, of 

u = arg min J(v), 
veV||,|| 

is a non increasing sequence: 

J(um) < J(u„j_i) for all m > 1. 
Moreover, if J{um) ~ J{um-i), Um-i is the solution of (prj) . 
Proof By definition, we have 

J{Um) < J(u„i_i + Zm) < J(u„i_i + Zm) < J(u„j_i + z) Vz € Si 

In particular, since £ iSi by assumption (Bl), we have J{um) < J(u„i_i). If J{um) = 
J{um—i), we have 

J{um-l) = min J{um-i + z) 
zeSi 

and by Lemma [5] we have that u^-i solves (j^. □ 

Remark 3 If J(um) = JivLm-l) holds for some m > 1, that is Um_i is the solution 
of then the updated PGD is described by a finite sequence of symbols a(u) = 
0.10.2 ■ ■ ■ OLm-i, where a.^ G {c, /, r} for 1 < fc < m — 1. Otherwise, { J(um)}meN is a 
strictly decreasing sequence of real numbers and a(u) G {c, i,r}'*. 

Definition 5 Let a £ {c, Z,r}. Then G {c, Z,r}'^ denotes the infinite sequence of 
symbols a. a. ■ ■ ■ a. ■ ■ ■ . 

From now on, we will distinguish two convergence studies, one with a weak conti- 
nuity assumption on functional J, the other one without weak continuity assumption 
on J but with an additional Lipschitz continuity assumption on the differential j' . 
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4.2.1 A first approach for weakly sequentially continuous junctionals 

Here, we introduce the following assumption. 
(A3) The map J : V||.|| — v R is weakly sequentially continuous. 

Theorem 4 Assume that J satisfies (A1)-(A3). Then every progressive Proper Gen- 
eralized Decomposition {um}m>li over Si, of 

u = arg min J(v) 

vev„.|| 

converges in V||.|| to u, that is, 

lim ||u — u,ri|| ~ 

Proof From Lemma |8] {J(um)} is a non increasing sequence. If there exists m such 
that J(um) = J(um_i), from LemmaO we have Um = u, which ends the proof. Let us 
now suppose that J(um) < J(u„i_i) for all m. J(um) is a strictly decreasing sequence 
which is bounded below by J(u). Then, there exists 

J* — lim J(um) > ^(u). 

If J* = J'(u), Lemma 13] allows to conclude that Um — )■ u. Therefore, it remains to 
prove that J* = J(u). Since J is coercive, the sequence {umjmgN is bounded in V||.||. 
Then, there exists a subsequence {umj.}j,grii that weakly converges to some u* G V. 
Since J is weakly sequentially continuous, we have 

J* = hm J(yimk) = J'(u*). 
By definition of the PGD, we have for all z G <Si, 

■^(UmCfc+l)) < Ji^ni^+l) < J^lUmfc + z) 

Taking the limit with k, and using the weak sequential continuity of J, we obtain 

J(u*) < J(u* +z) Vzg5i, 

and by Lemma [5] we obtain u* = u and a fortiori J(u*) — J(u), that concludes the 
proof. □ 

4.2.2 A second approach for a class of functionals with Lipschitz continuous 
derivative on bounded sets 

Now, assume that assumption (A3) is replaced by 

(A3) J' : V||.|| — > ^il ll Lipschitz continuous on bounded sets, i.e. for A a bounded 
set in V||.[|, there exists a constant Ca > such that 

||j'(v)-/(w)|| < C^||v-w|| (14) 

for all V, w G A. 

The next three lemmas will give some useful properties of the sequence {zm}m>l- 
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Lemma 9 Assume that J satisfies (A1)-(A2) and let {\im}m>l be a progressive Proper 
Generalized Decomposition, over Si, of 

u = arg min J(v). 

veV||.|| 

Then 

( j'(Um_i + Zm), Zm) = 0, 

for all m > 1. 

Proof Let z,n = Xm'Wm, with Am G and ||wm|| = L In the cases ctm = c (purely 
progressive PGD) and CKm — ^5 we have Zm — z-m G argminzg5j J(um_i + z). From 
assumption (B2), we obtain 

J(Um_i + A„iW„i) < J(u,„_i + AWm) 

for all A G R. This inequality is also true for ocm — r since Zm = arg min^guj^^^ ) J(u„i- 1 + 
z) and U(zrn) is a linear space. Taking A — Am ± 7, with 7 G R^, we obtain for all 
cases 

< - (J(Um-l + AmWm ± 7Wm) - J(Um-l + AmWm)) • 

7 

Taking the limit 7 \ 0, we obtain < ±(J'(um-i + AmWm), Wm) and therefore 

{j'(Um-l + AmWm), Wm) = 0, 

which ends the proof. □ 

Lemma 10 Assume that J satisfies (A1)-(A2). Then the corrections {zm}m>l of a 
progressive Proper Generalized Decomposition {um}m>l, over Si, of 

u = arg min J(v), 
veV||.|| 

satisfy 

oo 

llzmll** < oo, for some s > 1, (15) 

m— 1 

and thus, 

lim ||zm||=0. (16) 

m— >oo 

Proof By the ellipticity property we have 

J(Um-l) - J(Um-l +Zm) > (-j'(Um-l +Zm),Zm> + -||Zm||* 

s 

for some s > 1 and a > 0. Using Lemma |5] and J(um) < J(um-i + Zm), we then 
obtain 

J(Um-l) - J(Um) > -IIZmT (17) 

S 

Now, summing on m, and using limm-i-oo J(um) = J* < cx3, we obtain 

oo oo 

X] ^ - Ji^m)) = J(0) - J* < +00. 



Q 
S 



m— 1 m— 1 

which implies limm.->oo ItzmH*^ = 0. The continuity of the map x f-> x^^^ at a; = 
proves (|16p . □ 
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Lemma 11 Assume that J satisfies (A1)-(A3). Then for every progressive Proper 
Generalized Decompositions {urn}m>i, over Si, of u — argminvgVn y J'(v), there ex- 
ists C > such that for m > 1, 

KJ'(u„„i),z)| ^C||z,„||||z||, 

for all z G <Si . 

Proof Since J{um) converges and since J is coercive, {um}m>l is a bounded sequence. 
Since Hzmll — >■ as m — >■ oo (Lemma I10|l . {zm}m>i is also a bounded sequence. Let 
a > such that sup„ l|um|| + sup„ \\^m\\ < a, and let Cb be the Lipschitz continuity 
constant of j' on the bounded set B — £ ^\\-\\ ■ \W\\ — '^}- Then 

-(J'(u,„_i),z) = {j'(u„i_i +z) - j'(u„i_i),z) - (j'(u„i_i +z),z) 
<C7b||z|P-(J'(u,„_i+z),z> 

for all z G A = {z £ 5i : ||z|| < sup„ ||zm||}. By convexity of J and since J{um-i + 
Zm) < J{um-i + z) for all z £ <Si, we have 

(j'(Um_i + z), Zm ~ z) < J(u,„_i + Zm) - J(Um-l + z) < 

Therefore, for all z £ A, we have 

-{j'(Um-l),z) < Cb||z|P - (j'(Um-l +z),Zm> 

< Cb||z|P - (j'(Um-l + z) - j'(Um_i + Zm),Zm) (Lemmaig 

< Cisllzf + CB||z-Zm||||zm|| (Lemma^W^ 
<Cs(||z|p + |lz||||Zm|| + ||Zmf j 

Let z = w||zm|| G ^, with ||w|| = 1. Then 

KJ'(um_i),w)| < SCsllzmll Vw G {w G 5i : ||w|l = 1} 
Taking w = z/||z||, with z G Si, and C = 3Cb > we obtain 

Kj'(Um„i),Z>| < C||Zm||||z|| Vz G 5i 



Since V||.|| is reflexive, we can identify VJ|*|| with V||.|| and the duality pairing 

(■, •)v** V* with (■, ■)v* Vii II (i.e. weak and weak-* topologies coincide on Vn n). 

^ ' ' ^ II- II ' II -11 ^ ' ' ^ ||.|| ■ ^ II -11 ^ ^ ° il'lK 

Lemma 12 Assume that J satisfies (A1)-(A3). Then for every progressive Proper 
Generalized Decomposition {um}m>li over Si, of u = argminvgVn ^^ >^(v), the se- 
quence {j'(um)}meN weakly-* converges to in Vj|n , that is, limm— !.oo(J''(um), z) = 
for all z in a dense subset of 'Vu.u. 
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Proof The sequence {umjmeN being bounded, and since j' is Lipschitz continuous on 
bounded sets, we have that there exists a constant C > such that 

||j'(u™)|| = ||j'(u„0-/(u)|[ <C||u-u„,|| 

That proves that {j'(um)} C V|jn is a bounded sequence. Since is also re- 

flexive, from any subsequence of {j'(um)}mGN, we can extract a further subsequence 
{ J'(umj.)}/jgpj that weakly-* converges to an element (p G V|Jn. By using Lemma [TTI 
we have for all z G <Si, 

l(J''(UmJ,z)| < C|lz„j^+i||||z||. 

Taking the limit with k, and using Lemma [TOl we obtain 

(<^,z)=0 VzG<Si, 

By using assumption (B3), we conclude that ip — 0. Since from any subsequence of 
the initial sequence {,!' {um)}meN we can extract a further subsequence that weakly-* 
converges to the same limit 0, then the whole sequence converges to 0. □ 

Lemma 13 Assume that J satisfies (A1)-(A3) and consider a progressive Proper Gen- 
eralized Decomposition {um}m>li over Si, of u = argminvgVn n >^(v) such that for 
the ellipticity constant s of J and a(u) = ai • • • cxm ■ ■ ■ , one of the two following 
conditions hold: 

(a) s > 1 and there exists a subsequence {ctmfclfcgN such that ccmj. = I for all k > 1. 

(b) 1 < s < 2 and there exists k > 1 such that a(u) = ai ■ ■ ■ Q!j,_iQ!°° where ol G 
{c,r}. 

Then, there exists a subsequence {umtlfegN such that 

(j'(UmJ,UmJ 0. 

Proof First, assume that condition (a) holds. Recall that if OLm = I for some m > 1, the 
u,n is obtained by the minimization of J on the closed subspace U(um-i +Zm) C . 
The global minimum is attained and unique, and it is characterized by (J J'(um), — 
for all V G U(um_i + Zm)- Thus, under condition (a), there exists a subsequence such 
that (J'(um^), u,nfc) ~ for all fc > 1. Now, we consider that statement (b) holds. 
Without loss of generality we may assume that a(u) — a°° where ct G {c, r}. In both 
cases, Um = Yl'k^l ^k- Thus, we have 



m 



|(j'(Um),Um)| < l(j'(Um),Zfe) 



k=l 
m 

< ||z™+i||||zfe|| (By Lemma [m. 



A,-l 

Let s* > 1 be such that 1/s* -I- 1/s = 1. By Holder's inequality, we have 

l/s 



|(/(u„0,u„)| <C7||z,„+i|lml/^* (^i"^'"') 



k=l 



1/s 
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From Lemma lTOl we have X^^j^ INfc ll'' < oo- Then there exists a subsequence such that 
"^fc INmfc+l II* — >■ 0. For 1 < s < 2, we have s < s* . Since limi^^^ W^kW = 0, we have 
Nfcli'* < INfcll'' for k large enough, and therefore we also have m/j||zm;,+i||* — > 0, 
which from H18|) ends the proof of the lemma. □ 

Theorem 5 Assume that J satisfies (A1)-(A3) and consider a progressive Proper 
Generalized Decomposition {um}m>i, over Si, of u = argminveVy y J'(v) such that 
the ellipticity constant s of J and a(u) satisfy one of the following conditions: 

(a) s > 1 and there exists a subsequence {ctmfcjfcgM such that cxjru. ~ I for all k > I. 

(b) 1 < s < 2 and there exists k > 1 such that a{u) = oti ■ ■ Q:j,_]^a°° where a G 
{c,r}. 

(c) s> 1 and a{u) is finite. 

Then {um}m>lj converges in V||.|| to u, that is, 

lim ||u — Um|| = 0. 

m— >oo 

Proof From Lemma[Sl {J(um)} is a non increasing sequence. If (c) holds, there exists 
m such that J(um) = J(um_i) and from Lemma (8] we have Um = u, which ends 
the proof. Let us now suppose that J{um) < J(um-i) for all m. J{um) is a strictly 
decreasing sequence which is bounded below by J(u). Then, there exists 

J* — lim J(um) > J(u). 

m^oo 

If J* = J(u), Lemma |4] allows to conclude that {um} strongly converges to u. There- 
fore, it remains to prove that J* = J(u). By the convexity of J, we have 

J(Um) ~ J(u) < (j'(u„i),U„i - u) = (j'(u„i), U„i) - (J'(u„i),u) 

By Lemmas 1121 and 1131 we have that there exists a subsequence {um^j^-gm such that 
{J'(um^), Um^) — > and {j'(um^),u) — ^ 0, and therefore 

J* - J(u) = lim J(u,„J - J(u) < 
Since we already had J* > J(u), this yields J* = J(u), which ends the proof. □ 

5 Examples 

5.1 On the Singular Value Decomposition in spaces for p > 2 

A Banach space V is said to be smooth if for any linearly independent elements x,y £V, 
the function (j>{t) — \\x — ty\\ is differentiable. A Banach space is said to be uniformly 
smooth if its modulus of smoothness 

p(t) = sup 

lkll=lly||=i 



a; + rj/| 



\\x - Ty\\ 



- 1 



r > 0, 



satisfies the condition 
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In uniformly smooth spaces, and only in such spaces, the norm is uniformly Frechet 
differentiable. It can be shown that the L^-spaces for 1 < p < oo are uniformly smooth 
(see Corollary 6.12 in [57]). 

Following section 12.2.11 we introduce the tensor product of Lebesgue spaces 

with p > 2, and fi — fii ^ jj,2 a finite product measure. Recall that 

l|v|U, = \^{x)\Pd^^{x) 
\Jl1xI2 

Let u be a given function in L^{Ii x 12)- We introduce the functional J : x I2) 

M defined by 

j(v) = i||v-ur^^. 

Let G : Lfi{Ii x I2) — >■ K be the functional given by the p-norm G(v) = ||v||/ip. It is 
well known (see for example page 170 in \19\ ) that G is Frechet differentiable, with 

G'{v) = v\vr^M'-J' GLlihxh), 

with q such that 1/q+l/p — 1. We denote by C'^ the set of fc-times Frechet differentiable 
functionals from L^{Ii x I2) to R. Then, if p is an even integer, G € C°° . Otherwise, 
when p is not an even integer, the following statements hold (see [S] and 13.13 in [23|): 

(a) If p is an integer, G is (p— l)-times differentiable with Lipschitzian highest Frechet 
derivative. 

(b) Otherwise, G is [p]-times Frechet differentiable with highest derivative being Holde- 
rian of order p — [p] . 

(c) G has no higher Holder Frechet differentiability properties. 

As a consequence we obtain that G £ for all p > 2, and the functional J is also 
Frechet differentiable with Frechet derivative given by J'(v) = G(v — u)'°^^G'(v — u), 
that is, 

{j'(v),w)= / (v — u) |v — u|^ ^ vf dp.. 

Jll X I2 

Thus, J satisfies assumption (Al). It is well-known that if a functional F : V — > W, 
where V and W are Banach spaces, is Frechet differentiable aX v , then it is also 
locally Lipschitz continuous aX v £V. Thus, if p > 2, we have that J' G C^, and as a 
consequence j' satisfies (A3). 

Finally, in order to prove the convergence of the (updated) progressive PGD for 
each u G I/^(/i x h) over Si = 7(ri,r2)(-^m (-'"i) ®o ^i.2{h)), where {ri,r2) G N^, we 
have to verify that (A2) on J is satisfied. Since there exists a constant ap > such 
that for all s, i G M, 

{\s\P^^s - \t\P^^t){s ^t)> ap\s ~ t\P 

(see for example (7.1) in [S'), then, for all v,w G Lf^{Ii x I2), 

(j'(v) — j'(w), V — w) > ap\\v — w\f , 

which proves the ellipticity property of J, and assumption (A2) holds. 

From Theorem [SI we conclude that the (updated) progressive Proper Generalized 
Decomposition converges 
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— for all p > 2 if conditions (a) or (c) of Theorem [5] hold. 

— for p = 2 if condition (b) of Theorem [5] holds. 

Let us detail the application of the progressive PGD over 

We claim that in dimension d = 2, we can only consider the case ri = r2 ~ r. The 
claim follows from the fact that (see |14) 1 for each v G (Ji) ®a (^2), there exist 
two minimal subspaces ?7j_min(v), j = 1,2, with dim (7i njin(v) = dim(72,min(v) and 
such that V £ Ui^n^i^i'^) ®a t''2,min(v). In consequence, for a fixed r G N and for 

we let 

ui £ arg min ^J^fz). 

zGr(,,,-)(L2i(/i)®„L2,(/2)) 

Then there exist two bases {u^'^ , • ■ • , '"r'^} C -L^^ [Ij) of JTj.minCv), for j = 1,2, such 
that 

r r 

srsr (1)^ (2) 

ui = 2^ 2^ cTfc,; < ' ® % 
fc=l (=1 

and u — ui ^ (/i) ®a Lf^2 (^2)- Proceeding inductively we can write 

rnr mr 

At step m, an example of update of type ctm = r would consist in updating the 
coefficients {o-fc.; : k,l £ {(m — l)r- + 1, . . . , mr}}. An example of update of type ocm = I 
would consist in updating the whole set of coefficients {aj, i : k,l £ {1, . . . , mr}}. 

In the case p — 2 and when we take orthonormal bases, it corresponds to the 
classical SVD decomposition in the Hilbert space L^{Ii x I2). In this case we have 

Ed) (2) 

where aj = |(u, uf^ (gi u^^)\, for 1 < j < mr. 

In this sense, the progressive PGD can be interpreted as a SVD decomposition of a 
function u in a L^-space where p > 2. Let us recall that for p > 2, an update strategy 
of type (/) is required for applying Theorem [5] (at least for a subsequence of iterates). 

The above results can be naturally extended to tensor product of Lebesgue spaces, 

a ®fe=i ^fik ih) " with d > 2 and Si = TZi (^a^Slt^l ih) ) , leading to a gen- 
eralization of multidimensional singular value decomposition introduced in [15] for the 
case of Hilbert tensor spaces. 
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5.2 Nonlinear Laplacian 

We here present an example taken from [S]. We refer to section [2.2.2l for the introduction 
to the properties of Sobolev spaces. Let = J?i x . . . x Qj^. Given some p > 2, we let 
V||.|| = H^'^{n), which is the closure of C^{Q) (functions in C°°{Q) with compact 
support in fi) in H^ P{f2) with respect to the norm in H^'^{Q). We equip H^'^^Q) 
with the norm 

l/p 



M={Y.\\d.,{^)\\LHn^ 



which is equivalent to the norm || ■ ||i.p on H'^ P{n) introduced in section [2X2l We 
then introduce the functional J : V||.|| — > M defined by 

j(v) = i||v||f-(f,v), 

p 

with f G Its Frechet differential is 

j'(v) = A(v)-f 

where 



dx 



k=l 

A is called the p-Laplacian. Assumptions (A1)-(A3) on the functional are satisfied (see 
[S]). Assumption (B3) on the set TZi (^a ^j^i H^'^ {Qj)^ is also satisfied. Indeed, it 



can be easily proved from Proposition Othat the set TZi ^ a ^o'^(^j) j weakly 

closed in {H^'^{f2), \\ ■ Since the norm || • || is equivalent to || ■ on Hq'^{Q), 

it is also weakly closed in {Hq'^\Q), \\ ■ ||). 

Then, from Theorem[Sl the progressive PGD converges if there exists a subsequence 
of updates of type {I). 

5.3 Linear elliptic variational problems on Hilbert spaces 

■II 



Let V||.|| — Vi ®a ■ ■ ■ ®aVd be a tensor product of Hilbert spaces. We consider the 
following problem 

J(u) = min J(v), J(v) = ia(v, v) - ^(v) 

where K C a : V||.|| x V||.|| — !> R is a coercive continuous symmetric bilinear 

form, 

o(v, v) > a||v|p VvGV||.||, 

o(v, w) < /3||v||||w|| Vv,WGV||.||, 

£ : V||.|| — I- R is a continuous linear form, 

^(v)<7||v|| VvGV||.||. 
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Case where K is a closed and convex subset o/V||.||. The solution u is equivalently 
characterized by the variational inequality 

a(u,v - u) > £(v - u) \/v € K 

In order to apply the results of the present paper, we have to recast the problem as 
an optimization problem in V||.||. We introduce a convex and Frechet differentiable 
functional j : V||.|| R with Frechet differential j' : V||.|| — >■ V|n, such that j(v) = 
if V G A" and j (v) > if v ^ A". We further assume that j' is Lipschitz on bounded sets. 
We let je(v) = e~^j{v), with e > 0, and introduce the following penalized problem 

Je{Ue) = min Je(v), Je(v) = J(v) + je(v) 
vGV||.|| 

As e — >■ 0, je tends to the indicator function of set K and Ue — !> u (see e.g. |16p. 
Assumptions (A1)-(A2) are verified since Je is Frechet differentiable with Frechet dif- 
ferential j'e : V||.|| — !> Vj^.|| defined by 

(J^(v),z) = a(v,z)-^(z) + (j^(v),z), 

and Je is elliptic since 

{Je(v) - Je(w),V - w) = a(v - W, V - w) + {je{v) - ^^(w), V - w) > a||v - w||^ 

Assumption (A3) comes from the continuity of a and I and from the properties of j' . 

Case where K — V||.||. UK — V||.||, we recover the classical case of linear elliptic 
variational problems on Hilbert spaces analyzed in [15| . In this case, the bilinear form 
a defines a norm v a — \/a(v, v) on V||.||, equivalent to the norm || ■ [|. The functional 
J is here equal to 

J(v) = i||u-v||2-i||u||2 

The progressive PGD can be interpreted as a generalized Eckart- Young decomposition 
(generalized singular value decomposition) with respect to this non usual metric, and 
defined progressively by 

l|u - Um\\l = min |ju - u^-i - 
zeSi 

We have 
and 

||u - Umlla = ||u||a 

where am can be interpreted as the dominant singular value of (u — u„j_i) G V|| ||- 
The PGD method has been successfully applied to this class of problems in different 
contexts: separation of spatial coordinates for the solution of Poisson equation in high 
dimension OB], separation of physical variables and random parameters for the solution 
of parameterized stochastic partial differential equations |28) . 
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6 Conclusion 

In this paper, we have considered the solution of a class of convex optimization problems 

in tensor Banach spaces with a family of methods called progressive Proper General- 
ized Decomposition (PGD) that consist in constructing a sequence of approximations 
by successively correcting approximations with optimal elements in a given subset of 
tensors. We have proved the convergence of a large class of PGD algorithms (including 
update strategies) under quite general assumptions on the convex functional and on 
the subset of tensors considered in the successive approximations. The resulting succes- 
sion of approximations has been interpreted as a generalization of a multidimensional 
singular value decomposition (SVD). Some possible applications have been considered. 

Further theoretical investigations are still necessary for a better understanding of 
the different variants of PGD methods and the introduction of more efficient algorithms 
for their construction (e.g. alternated direction algorithms, ...). The analysis of algo- 
rithms for the solution of successive approximation problems on tensor subsets is still 
an open problem. In the case of dimension d = 2, further analyses would be required 
in order to better characterize the PGD as a direct extension of SVD when considering 
more general norms. 
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